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2.1 3D Input Devices 

3D input devices were developed to make as natural and 
intuitive an interaction possible for the user in 3- 
dimensional, computer-generated scenes. 3D input 
devices make it possible to carry out true 3D 
interactions, and thus offer disadvantages in principle 
over 2D input devices, with which 3D interactions must 
be re-perceived as sequences of 2D interactions. 3D 
input devices as a rule put six degrees of freedom at the 
user's disposal, namely three for the positioning and 
three for the orientation in space. 3D input devices may 
be divided into these three categories: 

'Flying' Devices 

Gesture-Controlled Devices 

Tabletop* Devices [74]. 

These three categories are briefly illustrated below, a 
relevant representative of each being presented and 
evaluated from the aspect of suitability for modeling 
assignments. Next, the requirements imposed by CAD 
on these devices will be discussed. 

2.1.1 Requirements for 3D Input Devices, in the 
View of CAD 

Within this section, the requirements that 3D-CAD 
imposes on 3D input devices will be formulated. In the 
analysis of requirements and evaluation, statements in 
[174], [144], [94] and [57] will be recalled and 
supplemented with aspects of 3D modeling [149]. 
The development of 3D input devices was advanced 
primarily in the field of "virtual reality." The aim of VR 
systems consists primarily in allowing the user to dive 
into an artificial, computer-generated world. The user 
(otherwise known in VR as a cybernaut) can manipulate 
his way through the virtual environment and manipulate 
objects by means of multidimensional interaction 
techniques. He can integrate with these objects in an 
"accustomed" manner, i.e. grasp them, move them and 
release them again. Upon entering the virtual world, as a 
rule the objects are already present (among the 
applications in VR, the modeling of objects represents an 
exception). A compact introduction to methods, 
techniques and fields of application of virtual reality is 
offered in [10] and [11]. 

Modeling functionality, such as the generation and 
modification of objects, is supported by VR systems in 
exceptional cases only. In contradistinction to VR 
systems, these are core functionalities of a modeling 
system. With a modeling system, it must be possible to 
generate and modify objects exactly, i.e. objects must be 
accurately dimensionable. Designers often work with 
their CAD system for several hours daily; therefore, 
different requirements are imposed on the fitness of the 
input devices for use than in VR. 3D input devices for 
modeling assignments should satisfy the following 
requirements: 
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They should be precise, i.e. points in space must 
be exactly determinable. 

Their radius of action must not be limited, since 
in principle, the construction space is unbounded. 

Rotation through angles of rotation of any size 
must be simple to execute. 

They must be ergonomic, to facilitate use over a 
fairly long period of time. 

They must not limit the user, so that he can 
continue to execute 2D interactions and/or keyboard 
inputs. 

They should give the user suitable "feedback" 
when force is applied, to facilitate operation. 

They should be economical compared to the 
costs of a CAD work station. 

2.1.2.1 The SpaceMouse - A Tabletop Input Device 

The SpaceMouse [98] is a tabletop 3D input device, 
making available six degrees of freedom (see Fig. 1). 
The SpaceMouse offers nine freely-programmable keys 
and a movable cap, realizing the six degrees of freedom. 
To deflect the cap, some force must be exerted. The 
cap of the SpaceMouse responds with a motion within 
narrow limits (±1.5 mm in translation, and ±4 degrees in 
rotation). This form of feedback gives the user a feeling 
for how far he has deflected the cap. 




Fig. 1 . The SpaceMouse [98] 

The quantities delivered by the SpaceMouse are relative 
position and rotation data. In other words, they indicate 
the translatory and/or rotational changes in the latest 
time interval. If the cap of the SpaceMouse is pressed 
uniformly in one direction, the corresponding graphic 
object will move accordingly. The greater the deflection 
of the cap of the SpaceMouse, the greater the resulting 
translation/rotation quantity, i.e. the faster the 
corresponding object will move. 
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2.1.3 The Polhemus Sensor - A Flying 3D Input 
Device 

"Flying" 3D input devices are devices attached to the 
user's hand or held in the hand and moved in space. 
The position and orientation of the input device and/or 
the hand in space is determined e.g. by means of an 
electromagnetic field. In the following, the Polhemus 
sensor [1] is described, representing one of the most- 
used flying systems. 

The Polhemus sensor, hereafter also called briefly 
Polhemus, is a 3D position and orientation sensor, used 
independently or in combination with other input devices, 
e.g. the data glove (see section 2.1.4). To determine 3D 
position and orientation, the Polhemus uses an 
electromagnetic coupling between a transmitter and a 
receiver. The transmitter is placed in a fixed position in 
space, while the Polhemus sensor, which represents the 
actual 'receiver,' can be moved in space freely. The 
absolute position and orientation of the sensor in the 
electromagnetic field emitted by the transmitter, is 
determined cyclically. For evaluation of the position and 
orientation, the sensor is connected by a cable to the 
computer, or an interface station, which performs the 
determination of position. 

2.1.4 The Data Glove - A Gesture-Based Input 
Device 

The Data Glove [10] in combination with flying 3D input 
devices, forms the classical input technology for VR 
systems. The Data Glove picks up the hand and finger 
movements of its user. From the placement of the 
fingers, manual gestures can be interpreted, which then 
trigger corresponding interactions. Typically, the Data 
Glove is used in combination with a position and 
orientation sensor (e.g. the Polhemus), which determines 
the absolute position and orientation of the hand in 
space. 

The recognition of the finger position by the Glove 
generally takes place by way of an optical measuring 
system. The Glove is provided on the upper side of the 
hand with thin glass fiber cables which span the fingers. 
One end of each glass fiber cable is provided with a 
luminescent diode and the other end with a 
phototransistor, which measures the arriving light of the 
diode. The curvature of a finger joint reduces the 
incoming light energy, which conveys information about 
the position of the finger. Conventional Data Gloves 
register up to 16 different finger positions. 

In VR systems, motions of the user's hand are often 
transmitted to a graphic echo in the form of a virtual 
hand. The virtual hand corresponds to the real motions, 
making possible a natural and intuitive navigation and 
manipulation. Within the 3D system, the gestures and 
motions of the user are followed by the virtual hand. 
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Through the accustomed course of movement, the user 
can grasp objects, move them and release them again, 
so the interaction is highly intuitive. The Data Glove is 
usually employed in immersive applications, that is, 
applications in which the user is fully involved in the 
representation of the computer-generated scene. 
Complete immersion is achieved by head mounted 
displays [42]. 

2.1 .5 Other 3D Input Devices 

Besides the input devices here described in more detail, 
the market offers a steadily increasing number of other 
3D input devices. Besides the SpaceMouse and the 
SpaceBall, and the various Data Gloves, 3D input 
devices with tactile feedback and devices with force 
feedback (Phantom [181]) or else wireless 3D input 
devices [112] are beginning to establish themselves. 
Beyond this, input devices are being developed for 
special applications, such as deformation of objects 
[103]. Other scientists are trying to improve possibilities 
of input without an explicit input device; mention may be 
made here primarily of video-based systems that pick up 
special markings and determine the position of these 
points in space [137] with the aid of image recognition 
methods. A survey of input device technology will be 
found under [32]. 

2.1.6 Evaluation and Selection of 3D Input Devices 

In the following, the input devices presented will be 
evaluated as to their suitability for the modeling process 
at the traditional workstation. It should be noted that the 
environment in which a 3D input device is to be 
employed affects the priorities of the above-mentioned 
requirements decisively and thus co-determines the 
nature of the preferred input devices. Thus, tabletop 
devices seem better suited to the conventional 
workstation. The Virtual Desk (cf. chapter 2.3.3), on the 
other hand, suggests the use of flying devices. Also, the 
Virtual Desk makes access to a normal keyboard more 
difficult, and other input modes (e.g. spoken commands) 
gain importance. 

The SpaceMouse exhibits very good ergonomics. Its 
cap is comparatively flat and can be conveniently 
grasped by the hand, so that hardly any manifestations 
of fatigue occur. The measuring mechanism of the 
SpaceMouse is simple, insensitive to manifestations of 
wear, and highly precise. Furthermore, the SpaceMouse 
is relatively economical compared to magnetic trackers. 
Thus, it fulfills the said requirements for a 3D input 
device for a conventional CAD work station in exemplary 
manner. A disadvantage of tabletop devices is that they 
will only indirectly allow 3D interactions (cf. section 2.2). 
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The advantage of flying input devices, namely the 
possibility of direct 3D interaction (cf. section 2.2) is at 
the same time its greatest disadvantage: The fact that 
they must be guided by the hand and arm in front of the 
body soon leads to fatigue. Another disadvantage is the 
usually necessary cabling of the user. The devices are 
not suitable for operation over a prolonged period. 
Besides, flying devices generally yield absolute space 
positions, i.e. the interaction space is predetermined. A 
'freezing mechanism' must be realized if the hand is to 
be movable without influencing the graphic echo, so that 
the graphic echo can be displaced out beyond the 
physical tracking region. 

Electromagnetic tracking systems exhibit the following 
disadvantages in view of the requirements of 3D 
modeling: 

The sensor is trouble-prone with respect to the 
influence of an outside field, so that even nearby metallic 
objects may interfere. 

The spatial resolution is limited, so that positions 
cannot be detected or computed with arbitrary precision. 

The reaction of the graphic feedback is in part 
delayed in time (owing to determination of position and 
noise suppression). 

The Data Glove does not permit changing the input 
device or even merely 'releasing' the input device in a 
simple manner, e.g. to write a brief note of something. In 
other words, it resists being imbedded in the typical 
course of a designer's work. Use of a Data Glove over a 
period of any length is therefore unreasonable for 
modeling assignments. In the following, other reasons 
are given that militate against the use of a Data Glove 
within a 3D CAD system: 

Typical CAD interactions, such as, for example, 
rotation through a large angle, are troublesome to 
execute with the Data Glove, since the hand is subject to 
natural limitations of movement; in the case of large 
angles, an embracing becomes necessary. 

The Data Glove must be individually calibrated to 
the user's hand in order to get satisfactory results. 

The Glove is fits very snugly, and this becomes 
unpleasant during prolonged wear. 

The lack of force feedback is found troublesome; 
as a rule, the user gets no tactile feedback in grasping. 

Different devices are required for right-handed 
and left-handed people. 

Some of the problems listed are solved or mitigated by 
new developments, e.g. the use of neuronal networks for 
gesture recognition [23] or by Gloves with tactile 
feedback (CyberTouch [182]) and force feedback 
(CyberGrasp [182]). However, experience with 
designers in the scope of user tests within the framework 
of this work allow the conclusion that the use of a Data 
Glove is not acceptable in daily work. In conclusion, 
here is a tabular comparison of the 3D input devices 
presented with respect to the formulated requirements. 
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Table 1: Comparison of Various 3D Input Devices as to Their 
Suitability for Use in Modeling Applications at the Conventional 
Work Station f991 ' 





Polhemus w/wo Mouse 
"Bat" 


Data Glove 


SpaceMouse 


Precision 




: 


++ 


Unlimited action 
space 


-- 


— 


++ 


Unlimited angle of 
rotation 






++ 


Ergonomy 






++ 


2D input still 
possible 






++ 


Feedback on 
application of force 






+ 


Intuitive interaction 


++ 


++ 


+ 


Price 






++ 



Legend: ++ very good, + good, - poor, / high, - very poor, / very high 



2.2 3D Interaction 

By 3D interaction is meant action of the user in 
computer-generated scenes. Here, a number of base 
(inter)actions may be distinguished: 

Navigation, i.e. targeted motion in the space, 
realized by changes of camera position and orientation. 

Positioning, i.e. placement of an object in a 
particular space position. 

Orientation, i.e. rotation of an object in space. 

Selection, and/or de-selection, i.e. choosing a 
point or object in space. 

Positioning and orientation are often taken together, 
since the two are assignments that can be performed 
simultaneously with a 3D input device having six degrees 
of freedom. 

From these base interactions, higher-grade interactions, 
such as moving of objects to a place, may be derived. 
For that purpose, first the graphic echo that represents 
the user in the scene (cursor or virtual hand) is taken to 
the object that is to be moved, and the object is selected. 
The object follows the ensuing hand or cursor 
movement, and is finally de-selected, or released. 

Within the course of interaction, direct manipulation [141] 
is an important concept and one of the basic 
requirements imposed on graphic interaction systems. 
Direct manipulation is when actions lead immediately to 
a visible effect. In the ideal case, the time interval 
between action and visual feedback is no longer than 
1/20 second. This corresponds to the rate of image 
repetition that is just sufficient to give the user the 
impression of a smooth motion. This ideal state is often 
not realizable, and investigations have shown that a time 
delay of up to one second is tolerated by users in certain 
applications. A delay of not more than one-tenth of a 
second should be a goal. 
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Another important concept is the so-called stimulus- 
response correspondence (or stimulus-response 
compatibility) [115], [59]. Stimulus-response 
correspondence is one of the fundamental requirements 
for intuitive man-machine intersection. 



upward rearward upward 

nacn ooen 




~- Beweeune der 2D-Maus „ ; , m \a^ c 

Screen (object upward) Motion of 2D Mouse Motion of 3D Mouse 

No Stimulus-Response Stimulus-Response 
Correspondence Correspondence 

Fig. 2. Principle of stimulus-response correspondence. 

A correspondence of stimulus and response is present 
when in response to a stimulus, e.g. a motion of the 
hand to the left, an object motion takes place in the 
same direction. In all interactions with real objects in the 
real world, stimulus and response correspond. In work 
with 2D use surfaces and the 2D mouse, on the other 
hand, there is no stimulus-response correspondence, 
since the mouse pointer moves upward on the screen, 
when the mouse is moved away from the user, 'rearward' 
(see Fig. 2). If there is no stimulus-response 
correspondence, then a mental mapping is required on 
the part of the user in order to execute the motion 
corresponding to the desired action. 

Stimulus-response correspondence for 3-dimensional 
assignments, such as volume modeling, can be 
achieved only with 3D input devices, and minimized 
outlay and scope of mental mapping operations. 

The concept of direct and/or indirect 3D interaction 
also becomes important in the course of the work, and 
will therefore be defined here. Direct interaction is 
present when the position of the user's hand coincides 
with the position of the graphic echo of the input device. 
Indirect interaction is present when the user's hand is not 
in the position of the echo of the input device. The 
conventional mode of operation with the 2D mouse is a 
form of indirect interaction, since the hand is not at the 
location of the mouse pointer. Operation with a Data 
Glove in an immersive environment, on the other hand, 
is for the most part in the nature of direct interaction. 



Page 8 of 20 



Docket No. 14XZ130599/GEM-0105 
Application No. 10/722,844 



[Translation from German] 

Excerpt from dissertation submitted by Andr§ 

Stork on 18 August 2000, Pages 80 - 89] 

4.2.6 Topology-Based Limited Modification 
Technique 

So far, interaction techniques have been presented to 
produce primitives, and/or modify objects by the 
production of primitives, as in the case of the implicit 
Boolean operations. At this point, the topology-based 
limited modification technique ("topological-context 
based modification," or TCBM, for short) is to be 
introduced, as a new interaction paradigm, to manipulate 
primitives and their parameters in graphic-interactive 
manner. This novel formulation makes use of the 
advantages of the 3-dimensional input with its six 
degrees of freedom for object modifications with reduced 
number of degrees of freedom, such as are typical in the 
CAD. These include: 

Modification of one or more object parameters 

Rotation of an object about an edge 

Translation of an object in a plane 
Liang and Green [94] developed the region based 
reshaping formulation, in which, as a function of the 
position of the 3D cursor, different object parameters 
could be modified. Region based reshaping has two 
serious disadvantages: 

In the case of complex objects, ambiguities 
result from overlap of regions with different significance 
for the manipulation. 

Interaction is not directly with the object, but with 
its periphery. 

The topology-based limited modification technique 
solves these problems. The fundamental idea is to 
modify objects as a function of the selected topological 
elements, the geometrical context, and the motion then 
executed with the 3D input device (gesture). The 
modification initiated by the gesture depends both on the 
object type selected and on the geometrical givens at the 
position selected. The possible modifications are limited 
on this basis, so that with adjoining cursor movements, 
only manipulations having a small(er) number of degrees 
of freedom are possible. The cursor movement is limited 
accordingly. It is to be noted that although in the further 
course of interaction, only modifications having few 
degrees of freedom will run, this new interaction 
technique necessarily presupposes the six degrees of 
freedom of 3D input devices and makes use of them in 
advantageous manner for object manipulations by 
means of natural, intuitive gesture; there is no switching 
between different modes of modification, e.g. between 
translation and rotation. Gesture recognition passes 
over seamlessly into direct manipulation (see section 
4.2.6.1). The principle of passing directly from a gesture 
into direct manipulation was presented by Rubine (125) 
for 2D gestures, and is here extended to 3D. 
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The term "gesture" is here understood as a translatory 
and/or rotational motion executed over a certain period 
of time by the user of a 3D input device. In the case of a 
tabletop input device, the gesture turns out to be the sum 
of the relative motions. This does not correspond to the 
meaning of the term gesture as used in VR systems, 
where as a rule, the term gesture is understood as a 
static attitude of the hand (in English, the term posture, 
which may be translated as Pose, best describes this 
situation). For gesture-based interaction, often elaborate 
operations, such as Neuronal Networks [24] or "Fuzzy 
Logic," are employed to compensate for the peculiarities 
of different users. 

Survey of topology-based limited modification procedure: 

1 . Select a topological element 

2. Execute a 3D gesture (rotation and/or translation 
of 3D input device) 

3. Recognize the gesture, in light of the relevant 
context 

4. If desired, represent a visual echo ("Center 
Shapes") to indicate the recognized gesture to the user 

5. Limit the cursor movement, if necessary 

6. Perform the direct manipulation 

7. End the modification 

8. Propagate the change through the system to the 
modeler 

The first three steps may be accounted to gesture 
recognition, or classification, which will be described in 
the following. 

4.2.6.1 Gesture Classification Phase 

A direct manipulative system requires answering times in 
the range from one-tenth up to at most one second, i.e. 
there cannot be a wait of arbitrary length until the motion 
executed by the user with the 3D input device is 
recognized as gesture. Also, gesture recognition should 
function as robustly and stably as possible, and coincide 
with the intention of the user. So the aim must be the 
development of a rapid, simple and robust gesture 
recognition and classification, functioning user- 
independently to a large extent. 

A gesture is classified, or recognized, with regard to the 
picked topological element and the local geometric 
features at the selected point. Local geometric features 
are considered to be normals, tangents and principal 
axes. In combination with tolerance thresholds, these 
features divide the design space into various parts, in 
terms of which the classification of the gesture executed 
is done. For this purpose, the results yielded by the 3D 
input device after the selection of a topological element 
are collected over a certain period of time and the 
resulting information is evaluated in terms of the 
classification spaces. Besides the spaces given by an 
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angle of tolerance, there is a threshold of rotation, 
transgression of which leads through the collected 
events to a rotation of the object. What interaction is met 
with after completion of gesture recognition depends on 
the picked object type (see chapter 4.2.6.2). 

The procedure of gesture recognition, simplified, may be 
described as follows: 

1 . The user picks a topological element 

2. The user executes a gesture 

3. Over a certain period of time, the translatory and 
rotational results triggered by the gesture are collected 

4. Now, a comparison is made to determine 
whether the rotational results have passed the rotational 
threshold, and if so, a rotation is initiated; otherwise, go 
on to 5 

5. With the aid of the translatory results, a new 
(virtual) cursor position is computed. 

This new cursor position is not visualized. With it, it is 
ascertained only in which partial space the cursor is 
located, and a corresponding translatory modification is 
initiated. 




surface 



Selection of a Topological Element 
Selection may concern a surface, edge or vertex. Fig. 43 
shows the topological elements and their geometrical 
context, in terms of which the gesture recognition is 
carried out. Specifically, at the particular position picked, 
these are: 

For a surface f, the normal n and the principal 
directions u and v 

For an edge e, the tangent in the case of a 
curve, or the edge itself 

For a vertex v, its position 
tangent 



Tangentc 




edges 



vertex 



43: Topological elements and their geometrical context 



In terms of two thresholds, which may be adapted by the 
user, we now check whether a rotary motion or a 
translatory motion is present (see Fig. 44); this is most 
clearly illustrated by the example of an edge. 
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Definitions (cf. Fig. 44): 

Let e be an edge (e will be understood in the following 
as a vector) 

c 0 is the cursor position on the picked edge at gesture 
recognition time 0 

q is the cursor position at a point in time / 
r is the translation threshold angle 

g is the gesture vector; equivalent to the 'rubber band' vector during 

object production 

p is the rotation threshold angle 

cpi is the angle of rotation on the 3D input device at a 

point in time / 

n is the corresponding angle of rotation 



e 




gesture vector g 

Fig. 44: Threshold angle for classification of modification gesture 



In combination with e, t forms a partial space. If the 
cursor, after the gesture recognition period, lies in this 
space, ail further cursor movements lead to translation of 
the object belonging to e in or against the edge direction. 
Another partial space is defined by t and the plane for 
which e represents the normal to the surface. This 
partial space, for the sake of clarity, is not shown in Fig. 
44. The interpretation of the partial space is object-type 
dependent (see chapter 4.2.6.2). 

Time Interval for Recognition of Gesture 
To ensure a system response after at most one second, 
this time interval is defined as an upper limit, within which 
a gesture must be recognized. Events for gesture 
recognition are collected until a certain number of events 
are present or the time barrier is past (in which case only 
the events collected so far are taken into account, which 
may adversely affect the recognition rate). In user tests, 
12 events has crystallized out as the minimum for an 
input device operating at 50 Hz, such as e.g. the 
SpaceMouse. Then 12 events corresponds more or less 
to one-quarter second. Users not practiced in handling 
the SpaceMouse will, at the beginning of the interaction 
— while moving the cap in the desired direction — 
produce events that do not clearly go towards the 
intended gesture. For this reason, all events arriving 
within the first 0.2 second after the start of time 
measurement are ignored by gesture recognition. Thus, 
as a rule, a system response is assured after about half 
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a second (see Fig. 45). In the course of this interval z 
both the translatory and the rotational events of the 3D 
input device are picked up and processed as described 
below. 



beginning of interaction ✓ seamless transition 

ignored phase gesture recognition phase modification phase 



Ignoranzphasc 


Gestenerkennungsphase 


Modifikationsphase 


0 • o, 


2 


0, 


»» 

5 



*Zeit [sek] 



time [sec] 

Fig. 45: Seamless transition of gesture recognition after modification 



Calculation of Rotation and Translation 
The rotation delivered by the 3D input device at any time 
/ may be taken as axis and angle of rotation. For gesture 
recognition, in particular rotations about the axis or axes 
defined by the picked point, such as e.g. edge, normals 
to surface/principal directions, are relevant. Rotations 
whose axis is orthogonal thereto are not to play any part, 
and here again the stimulus-response correspondence is 
taken into account. 

This means that the rotations over the gesture 
recognition period z for each event, i.e. at each time /, 
are imaged on the reference axis, and the angle of 
rotation about this axis must be determined. To obtain 
the angle with respect to this axis, the axis obtained from 
the gesture (understood as a standardized directional 
vector) is imaged on the reference axis about which 
rotation is to occur. The angle of rotation is multiplied by 
the length of the projection. As a result, the relevant 
angle of rotation will be smaller, the more closely the 
angle a between gesture vector and reference axis 
approaches 90 degrees (see Fig. 46). 

Ti standardized gesture axis 



c 




r * * 4 > 



<ft gesture angle 
picked edge e axis projected on edge pro rata angle 

in edge direction e [0, <p] 

Fig. 46: Imaging of gesture rotation on axis of rotation 
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The total angle of rotation O is thus given by: 



The procedure described ensures kinesthetic 
correspondence [115] in 3D. That is, the user must 
rotate the 3D input device about the axis around which 
the picked object is to be rotated, just as he would rotate 
a real object in his hand about the axis around which he 
would like to rotate it. Thus, a stimulus-response 
correspondence is produced for rotation as well. 

Calculation of the translation executed by the 3D cursor 
over the gesture recognition period is comparatively 
trivial. It may be pointed out again that during this period, 
the 3D cursor changes its position virtually only, not 
visually. 

The gesture vector turns out to be equivalent to the 
rubber-band vector: 
g^-co 

Classification of Gesture 

The decision as to whether the user has executed a 
rotation gesture or a translation gesture is made finally 
by comparison with the thresholds, rotation having 
priority over translation. This has proved sensible, 
because a rotation of a 3D input device is always 
attended by a translation. The motor abilities of man do 
not permit performance of a rotation alone; some - even 
if not much -- translatory motion is always present. The 
mechanical and sensory properties of 3D input devices 
contribute their share to the circumstance that rotations 
cannot be executed wholly without translatory 
components. 

Therefore: 

If 0 > p, then execute rotation. 
If 



then execute translation or change of parameters. 
Otherwise, reject gesture. 

This completes the gesture classification phase. The 
events picked up to find the gesture are not applied to 
the object, i.e. the object remains intact up to the present 
time. To indicate the recognized gesture to the user, a 
visual echo is inserted into the scene (see Fig. 48). Now 
the modification phase begins, in which direct 
manipulation of the picked object takes place according 
to the rules of gesture interpretation described in the next 
subchapter. 
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4.2.6.2 Modification Phase 

When the gesture recognition phase is completed, the 
TCBM passes seamlessly into the modification phase, 
while the user is continuing his gesture. At the transition 
from gesture recognition to modification phase, a visual 
feedback may be inserted into the scene, and the 
freedom of motion of the cursor limited for all further 
events. During the modification phase, not only is the 
object altered in directly manipulative manner, but, with 
each event, the position or perhaps the image 
manifesting the visual feedback is adapted. 

Upon viewing of various types of volume primitives, it 
soon becomes clear that one and the same gesture 
applied to different primitives is not necessarily able to 
lead to a similar modification. Thus, the following 
modifications are to be supported by only two kinds of 
gestures (translation and rotation): 
Change of a parameter 

Simultaneous change of several parameters 
(scaling) 

Rotation of the object 
Translation of the object 

For this reason, a context-dependent interpretation of the 
gesture is required, coinciding as best may be with the 
user's intention. Like the interaction techniques for 
producing primitives, the rules of modification must also 
work correctly in semantic terms. Here, the volume 
primitives should behave as though they consisted of 
rubber, a modification being capable of functioning 
locally. For example, a displacement of the lid surface of 
a cylinder should not necessarily be attended by a 
modification of the base surface. 

The interpretation of modification gestures in a manner 
conforming to expectation was determined 
experimentally in terms of a prototypical transformation 
for the rectangular solid and cylinder as primitives. The 
first column contained the picked topological element. In 
column 2, the gesture is represented in terms of the 
particular object type in the model coordinate system 
(MKS) of the object. The third column gives the rule of 
transformation, and the fourth column, any exceptions to 
that rule. Note that in Table 4, all translatory gestures 
leading to change in one or more parameters of an 
object are expressed in terms of scaling. This form of 
representation was chosen to simplify the mathematical 
description. Actually, such translatory gestures are 
imaged on parameter changes of the volume primitive. 

Definitions and Abbreviations 

s scaling vector before modification 

m modification vector, found from gesture vector 

after application of the restriction of motion for the cursor 

and projection into MKS; in Table 4, m is represented as 

an arrow 

n normal vector 
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opp opposite picked point the point given by double 
the vector of the pick position to the center of the 
primitive 

cobf center of bottom face 

center of bottom surface 

copf center of opposite picked face 

center of surface opposed to the planar surface adjacent 

to the picked edge 

origin zero point for scaling 

\v\ vector, standardized with respect to v 

\\v\\ length/absolute value of a vector v 
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Table 4. TCBM modification rules 



Topologica 
I Element 


Gesture 


Transformation Rule 


Exception 


surface 


orthogonal to normal 


Translate object in plane 




planar 
surface 


direction of normal 


s' = s + m 
origin = opp 




Non-planar 
surface 


direction of normal, 
n 2 =0 


s' = s + m 
origin = cobf 


sphere: 

cursor limited to 
n 

*'i=5i+<(M*»)*IM>' 

i€x,y,z 

origin = center of 
sphere 

a 


vertex 


Rotation 


rotate obiect about vertex 






Translation 


s' = s + m 
oriain = odd 


cone: cursor 

confined 

to z-axis 

A 


edge 


direction of 

pHnp/tannpnt 
cuyc/ idi lyci u 


Translate object along line 






rotation about edge 


rotate object about edge/tangent 




circular 
edge 


direction of vector of 
curvature 

U 


5' iS =ii+((lm|*«) * ND»i€x,y 
origin = cobf 






onnogonai to vector ot 
curvature 


j' x =j 2 + m z 

j' = J xv + ((Kyi * "ay) * IKylD 



origin = copf 




elliptical 
edge 


direction of vector of 
curvature 


s' = s + m 
origin = cobf 






orthogonal to vector of 
curvature 


s* = s + m 
origin = copf 
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Prismatic object shapes may be treated like the 
corresponding conical shapes. Then horizontal edges in 
the model coordinate system must be distinguished from 
vertical ones to achieve an interpretation conforming to 
expectations; likewise peripheral, bottom and top 
surfaces. 



An example will clarify the mode of operation of the 
TCBM. Fig. 47 shows how a rectangular prism can be 
modified by various gestures after picking the same 
position in different ways. In Fig. 47a, the 3D cursor, 
after selection of the edge, is moved in the direction of 
that edge. At expiration of the gesture recognition 
period, the cursor lies in the compartment given by the 
edge direction and the translation threshold angle 
(visualized in Fig. 47 by a semi-transparent envelope^). 
Now a visual echo is inserted (see Fig. 48a), which 
indicates the recognized modifying intention to the user. 
All subsequent instructions are imaged on and restricted 
to the straight line defined by the edges. The rectangular 
prism can be moved only along the straight line. In Fig. 
47b, following the gesture recognition period, the cursor 
is located in the compartment defined by the surface 
orthogonal to the edge and the translation threshold 
angle (likewise visualized as semi-transparent 
compartment). Now the visual feedback represented in 
Fig. 48b is inserted. All subsequent interactions are 
imaged on the surface orthogonal to the edge passing 
through the picked point. The prism can now be scaled 
in two directions. 



picked point tolerance angle edge direction 

Toleranzwinkel 
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2 This space is open in the direction of the floor surface, of the cone used for 
visualization 
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unlimited compartment 

unbeschrankter 
Teilraum 



picked point 

edge direction 

antcnric 




position 

ennungszeii 



cursor position after recognition period 

b) scaling of a prism 

Fig. 47: Example for topology-based limited modification technique 





a) translation along edge 

Fig. 48: Visual feedback during TCBM 



b) scaling of two parameters 



The visual feedback, in the case of translation along a 
line, consists of a double arrow (see Fig. 48a), 

so that the user will realize that he can displace the prism 
in two directions. In the case of scaling in two 
dimensions, the graphic echo consists of a transparent 
surface orthogonal to the edges at the picked position. 
Two bi-directional arrows indicate the degrees of 
freedom (see Fig. 48b). 



It may be pointed out once again that during the 
modification phase, the changes are carried out, i.e. 
calculated and visualized, only by the graphic manager. 
After conclusion of the modification phase, the change is 
transferred to the mathematically correct object 
representation in the modeler (cf. chapter 3.3). In this 
way, a direct manipulation in real time is possible. As in 
the case of object production, the interaction may also be 
discretisized at the time of modification, so that precise 
manipulations can be performed (see chapter 4.2.9). 
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